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This  paper  presents  a  generic  approach  for  selecting  sustainability  criteria  and  indicators  (C&I)  by  using  a 
participatory  methodology.  Selecting  appropriate  C&I  to  assess  the  sustainability  of  projects  or 
developments  is  crucial  and  significantly  influences  the  assessment  results.  The  methodology,  which 
consists  of  two  processes:  a  pre-selection  of  suitable  C&I  by  the  researchers  and  a  final  selection  by 
regional  bioenergy  experts  in  a  multi-stakeholder  forum,  was  applied  in  a  Scottish  region  (Tayside  &  Fife). 

The  paper  concludes  that  the  methodology  provides  a  transferable  approach  to  systematically  select 
C&l  and  to  justify  this  selection.  Furthermore,  the  pre-selected  and  final  C&I  for  the  case  study  can  serve 
as  a  benchmark  set  and  starting  point  for  similar  decision  making  situations  in  other  regions.  In  order  to 
provide  a  wide  application  potential  for  the  bioenergy  field,  the  case  study  took  account  of  a  range  of 
scenarios  and  alternatives.  Moreover,  the  identified  C&I  and  requirements  for  selection  from  literature 
provide  a  broad  application  potential  to  assist  C&l  selection,  as  they  cover  the  sustainability  field  in 
general,  the  energy  sector  and  the  bioenergy  sector  in  particular. 
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1.  Introduction 

This  paper  presents  a  methodology  for  participatory  selection  of 
criteria  and  indicators  (C&I)  for  bioenergy  developments.  The  meth¬ 
odology  was  applied  in  a  case  study  area  in  Scotland  (Tayside  and  Fife). 
Sheppard  and  Meitner  [1]  and  others  [2-4]  highlight  that  C&I  reflect 
issues  of  concern  of  stakeholders  and  experts  of  an  industry  and  can 
be  used  to  measure  and  communicate  the  sustainability  of  projects  or 
progress  on  sustainable  development.  In  the  ‘Guidelines  for  Applying 
Multi-Criteria  Analysis  to  the  Assessment  of  Criteria  and  Indicators'  [5], 
C&I  are  stratified  into  four  levels.  The  first  level  comprises  of  a 
principle,  which  is  described  as  a  “fundamental  tmth  or  law  as  the 
basis  of  reasoning  or  action”.  Principles  justify  the  chosen  criteria, 
indicators  and  verifiers.  On  the  second  level,  a  criterion  is  defined  as  a 
“principle  or  standard  that  a  thing  is  judged  by”.  Criteria  enhance  the 
meaning  and  operability  of  principles,  however  do  not  measure 
performance.  On  the  third  level,  an  indicator  is  described  as  a  variable 
or  component  “used  to  infer  the  status  of  a  particular  criterion”. 
Indicators  are  comprised  of  specific  information  about  a  criterion,  and 
can  be  measured  by  verifiers,  which  are  defined  as  “data  or  informa¬ 
tion  that  enhances  the  specificity  or  the  ease  of  assessment  of  an 
indicator”  [5]  (p.ll  et  seq.).  Waage  et  al.  [6]  and  Madlener  et  al.  [7]  use 
a  similar  hierarchy  by  describing  that  criteria  are  operationalized  with 
indicators.  For  example,  the  criterion  ‘climate  change'  can  be  oper¬ 
ationalized  by  the  indicator  ‘C02  equivalent  emissions’.  Flowever,  in 
literature  about  decision  making  it  can  be  observed  that  the  term 
‘indicator’  is  sometimes  synonymously  used  with  ‘criterion’  and  vice 
versa  [8,9]. 

Vera  [3]  stress  that  if  taken  together  C&I  can  provide  an 
overview  of  the  whole  energy  system,  including  interlinkages 
and  trade-offs  among  various  aspects  of  sustainable  development, 
as  well  as  long-term  implications  of  current  decisions  and  beha¬ 
viors.  Progress  or  lack  of  it  towards  sustainable  development  can 
be  measured  by  observing  changes  in  C&I  values  over  time  [10], 
which  allows  the  carrying  out  of  focused  measures  for  imp¬ 
rovement  to  avoid  adverse  effects  in  specific  areas,  if  required. 
Furthermore,  they  can  be  appropriate  tools  for  communicating  and 
promoting  dialogue  with  respect  to  energy  issues  related  to 
sustainable  development  to  stakeholders,  policymakers  and  the 
public  [3],  Additionally,  Olsthoorn  et  al.  [11  ]  point  out  that  the  high 
complexity  level  of  an  issue,  for  instance  the  natural  environment, 
demands  the  need  of  appropriate  C&I.  The  selection  of  appropriate 
C&I  is  crucial  to  assess  a  project  or  development,  because  this 
choice  significantly  influences  the  results  of  an  assessment  [12]. 
Therefore,  it  was  emphasized  that  in  the  sustainability  context,  C&I 
selection  becomes  a  “delicate  process  of  translation  from  socio- 
environmental  agreements  to  specific  observed  properties  of  a 
complex  system”  [13]  (p.  2562),  and  that  the  chosen  C&I  are  the 
result  of  the  social  and  political  framework  at  a  particular  period  in 
history  [14]. 

In  this  paper  a  methodology  for  participatory  selection  of  C&I  for 
bioenergy  developments  is  described.  The  methodology  was  also 
applied  in  the  mentioned  case  study  area  (Tayside  &  Fife).  First, 
literature  on  methodologies  to  select  C&l  for  decision-making  in  the 
sustainability  assessment  field  in  general,  as  well  as  the  energy  and 
bioenergy  fields  in  particular,  was  reviewed.  Then,  an  appropriate 
methodology  for  participatory  C&I  selection  was  carried  out  starting 
with  identifying  C&I  from  literature  against  which  sustainability  in  the 
bioenergy  and  the  wider  energy  sector  have  previously  been  assessed. 
This  was  followed  by  the  identification  and  discussion  of  requirements 
to  select  appropriate  C&I  from  literature.  Based  on  these  two  generic 
steps,  the  methodology  was  specifically  applied  in  the  case  study  area, 
which  involved  multiple  steps.  First,  scenarios  and  alternatives  were 
developed.  Second,  most  suitable  requirements  for  C&I  selection  for 
the  case  study  were  determined.  Then,  the  sustainability  C&I  identified 
from  literature  were  pre-selected  in  accordance  with  the  determined 


requirements.  The  next  step  was  to  present  the  resulting  short-list 
of  C&I  to  identified  bioenergy  experts  for  discussion  in  a  multi¬ 
stakeholder  forum.  In  this  forum  the  participants  were  asked  to  reach 
a  consensus  on  the  most  suitable  C&I  for  the  case  study  by  taking 
account  of  the  same  requirements  for  selection  as  in  the  pre-selection 
process.  In  this  paper,  the  results  of  the  application  of  the  participatory 
C&I  selection  methodology  are  also  discussed  before  conclusions 
are  drawn. 

2.  Methodology  for  participatory  C&I  selection 

In  this  section  methodologies  to  select  C&I  for  decision-making  in 
the  sustainability  field  and  the  energy  and  bioenergy  sectors  from 
relevant  literature  are  briefly  described.  In  general,  Starkl  and 
Brunner  [15]  recommend  that  integrative  assessments  of  sustain¬ 
ability  should  not  be  based  on  prescribed  or  standardized  C&l  and 
rather  be  specifically  selected  on  a  case  to  case  basis.  Due  to  the  sheer 
volume  of  C&I,  which  can  be  found  in  literature,  several  authors  of 
academic  papers  propose  to  identify  and  pre-select  C&I  based  on 
relevant  literature,  followed  by  a  process  of  verification  and/or 
refinement  by  stakeholders  [1,16,17].  Gilmour  et  al.  [18]  followed 
this  approach  and  additionally  developed  new  case-specific  C&I. 
First,  the  C&I  pre-selection  process  took  place.  Then,  key  stakeholders 
were  individually  interviewed  to  verify  and  refine  pre-selected  C&I. 
In  this  process  stakeholders  were  asked  to  take  account  of  the 
specific  drivers,  aspirations  and  objectives  of  the  development. 
In  addition  to  the  interviews  a  document  analysis  to  refine  and 
verify  potential  indicators,  as  well  as  to  identify  associated  data 
availability  was  undertaken. 

Sheppard  and  Meitner  [1],  Gilmour  et  al.  [18]  and  Graymore 
et  al.  [16]  suggest  using  C&I  or  selection  frameworks,  which  are 
specific  to  a  countiy  or  region  and/or  which  are  well-established  in 
the  industrial  sector.  For  instance,  in  Graymore  et  al.  [16],  a  project 
undertaken  in  South-West  Victoria  (Australia)  is  described,  for 
which  a  set  of  C&I  applicable  for  tracking  progress  towards 
sustainable  development  was  identified.  First,  they  identified  a  vast 
array  of  C&I  already  in  use  for  measuring  sustainability  at  a  global, 
regional  and  local  scale  through  an  extensive  literature  review. 
Using  this  ‘global’  set  of  C&I  a  ‘filtering’  process’  was  carried  out  to 
refine  the  C&I  set  by  involving  stakeholders  representing  regional 
organizations  and  communities  from  their  case  study  area. 

Following  the  methodology  recommendations  from  the  reviewed 
literature,  the  methodology  presented  in  this  paper  consisted  of  both, 
generic  and  case-study  specific  parts.  First,  C&I  from  literature  were 
identified  against  which  sustainability  in  the  bioenergy  and  the  wider 
energy  sectors  have  previously  been  assessed  (Section  2.1).  Then, 
requirements  to  select  appropriate  C&I  from  literature  were  identified 
(Section  2.2).  Based  on  these  two  generic  steps,  the  methodology  was 
specifically  applied  in  the  case  study  area  involving  the  main  steps: 
development  of  scenarios  and  alternatives  (Section  3.1),  selection  of 
requirements  for  C&I  selection  (Section  3.2),  C&I  pre-selection  by  the 
researchers  (Section  3.3)  and  final  C&I  selection  by  regional  bioenergy 
experts  (Section  3.4).  In  the  subsequent  sections  these  processes  are 
described  and  discussed  in  detail. 

2.3.  Identification  of  sustainability  C&I  in  literature 

Employing  C&I  for  decision  making  has  been  extensively  addressed 
in  academic  papers  and  other  publications  covering  the  sustainability 
assessment  field  in  general,  as  well  as  the  energy  and  bioenergy  fields 
in  particular.  In  Table  1C&I  from  reviewed  literature  are  summarized. 
In  general,  it  can  be  observed  that  C&I  in  literature  are  either  grouped 
into  the  broad  categories  of  social,  economic  and  environmental 
[2,19-22]  or  an  additional  forth  aspect  (technical)  is  common  to 
evaluate  energy  systems  or  scenarios  in  particular  [7,9,12,17,23,24], 
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Table  1 

Summary  of  C&I  identified  in  literature. 


C&I  related  to: 


Source: 


Environmental  C&I: 

N0X  emissions 

Air  quality  or  other  non-GHG  emissions  (e.g.  CO  emissions,  S02  emissions,  particles  emissions) 

C02  emissions 

GHG  emissions  or  climate  change  (e.g.  total  carbon  savings) 

Land  take,  use  or  requirement,  land  use  change  or  effective  land  use 

Direct  or  indirect  land  conversion  caused  by  biofuel  plantation  (e.g.  visual  impacts,  quality  of  landscape,  effects  on  biodiversity  and 
ecosystems,  species  protection,  deforestation,  soil  quality  and  protection,  crop  management  and  agrochemical  use) 

Ecological  justice 

Adaptability 

Noise 

Water  consumption 

Water  management  or  quality 

Waste  generation  or  management 

Rational  use  of  resources  or  natural  resource  efficiency 

Economic  C&I: 

Investment  costs 

Operational  or  maintenance  costs 

Costs  and  economic  viability  (e.g.  net  present  value  (NPV),  payback  periods,  life  cycle  cost,  microeconomic  sustainability) 

Energy  prices/costs  for  end-users 

Macroeconomic  sustainability,  economic  development,  stability,  benefit  or  output,  balance  of  trade  or  payments 
Technological  advantage  or  diversification  of  technologies 

Resources  or  fuel  availability  or  import  dependency  (e.g.  import  dependency,  resource  availability,  regional  self-determinacy,  energy 
security,  reliability  of  electricity  supply) 

Technical  C&I: 

Efficiency 

Health  or  safety  of  energy  systems 

Maturity  or  knowledge  level  of  the  energy  generation  technology  in  use 
Reliability  of  energy  systems  as  well  as  their  technological  capability  and  limitations 
Electricity  generation  potential  or  average  annual  availability 
Primary  energy  ratio 

Energy  intensities  of  the  industrial,  household  and  commercial  level  (e.g.  industrial,  agricultural,  service/commercial,  household  and 
transport  energy  intensities) 

Energy  balance 

Social  C&I: 

Job  or  employment  creation  or  opportunities 
Job  income  or  working  conditions 

Access  or  rights  to  use  land,  water  and  other  natural  resources 
Food  security  in  the  context  of  bioenergy  generation  and  biofuel  supply 

Decision-making,  participation,  responsibility,  confidence  or  empowerment  as  well  as  planning,  management  or  monitoring 
Wider  social  C&I  (e.g.  respect  for  human  rights  and  minorities,  standard  of  living) 

Accessibility,  affordability  or  disparities 

Social  or  cultural  cohesion,  acceptability  or  benefits  (e.g.  social  justice,  effect  on  public  spending,  social  impacts) 

Compliance  with  laws 


[20,24,27,30-33] 

[3,7,8,12,17,19,24,31] 

[8,12,17,20,24,26,27,30,32- 

38] 

[3.9.19.25.31.39] 

[12.24.38.40] 

[3.7.9.17.19.39] 

[7,17] 

[7.17] 

[7,17,19,24,41] 

[9.12.38] 

[3.7.17.19.39] 

[3,18,19,39,41 ,42] 
[7,8,17,19] 

[8,12,18,20,24,25] 

[7.8.17.20.24.38] 

[19.24.25.37.40] 

[3,9,20,24] 

[17.19.39.40] 

[3.7.17] 

[3,7,9,12,17,38,40,43] 


[3,8,9,12,20,24-29,38] 

[3,24,39] 

124,25,34,35] 

[24,28,30,44] 

[12,31] 

[24,34,35] 

[3,20] 

[19,42] 


[7,8,12,17- 

20,24,25,27,28,30,31,35] 

[12.19.20.39] 

[19.39] 

[19,39] 

[17-19] 

[19,39[ 

[3,20] 

[7,9,17-19,24] 

[19[ 


The  identified  C&I  (Table  1 ),  which  built  the  basis  for  the  case  study’s 
C&I  pre-selection  (3.3),  are  also  divided  in  these  four  categories 
following  the  alternative  ordering  of  “environmental”,  “economic”, 
“technical”,  and  “social”  aspects. 

In  sustainable  energy  decision  making  it  can  be  observed  that 
‘C02  emission’  and  ‘Investment  cost’  are  the  most  common 
environmental  and  economic  aspects,  respectively  [24],  Further¬ 
more,  ‘Efficiency’  is  the  most  common  technical  criterion  to 
evaluate  energy  systems  according  to  Jovanovic  et  al.  [20]  and 
others  [8,24-29]  and  in  their  evaluation  of  the  impacts  of  power 
plants,  Chatzimouratidis  and  Pilavachi  [30]  state  that  in  the  energy 
sector,  ‘Job  creation'  is  considered  as  the  key  social  aspect  used  for 
sustainability  assessments. 

It  can  be  concluded  that  the  vast  number  of  C&I  makes  it 
imperative  to  take  a  systematic  approach  to  select  C&I  for  individual 
projects  or  case  studies.  A  systematic  approach  using  requirements 
for  C&I  selection  is  described  in  the  subsequent  sections  of  this  paper. 

2.2.  Requirements  for  criteria  and  indicators  selection  from  literature 

Requirements  for  C&I  selection  enable  a  ‘filtering  process’  as 
mentioned  previously.  These  requirements  can  be  found  in  literature 


about  the  sustainability  assessment  field  in  general,  as  well  as  about 
the  energy  and  bioenergy  fields  particularly.  These  types  of  literature 
were  considered  in  order  to  select  the  most  appropriate  set  of 
requirements. 


2.2.3.  Requirements  for  C&1  selection:  sustainability  assessment  field 

Requirements  for  sustainability  C&I  selection  have  been  widely 
described  in  literature  about  sustainability  assessment  in  general. 
Graymore  et  al.  [16]  used  three  ‘filters’  ensuring:  (a)  C&I  relevance  to 
the  region,  (b)  consideration  of  both  trend  and  condition  (time 
dynamic  C&I)  and  (c)  consideration  of  the  relationships  between  C&I 
to  avoid  an  isolated  perspective  on  C&I.  First,  they  asked  stakeholder 
organizations  and  community  representatives  to  select  most  suitable 
C&I  from  a  set  of  pre-identified  C&I  considering  regional  relevance 
(first  filter).  Then,  the  researchers  filtered  the  remaining  C&I  with 
respect  to  practicality  and  reliability  and  CIS  software  applicability 
(second  filter).  Finally,  based  on  this  reduced  set  of  C&I  the  third  filter 
took  relationships  and  indicator  independencies  into  account  to 
develop  a  small  number  of  highly  influential  C&I. 

Singh  et  al.  [21]  provide  the  following  dimensions  of  C&I 
measurements:  (a)  sustainability  aspect,  (b)  technique/method 
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employed  (quantitative/qualitative,  subjective/objective,  cardinal 
/ordinal,  one-dimensional/multidimensional),  (c)  type  of  compar¬ 
ison  (across  space  (cross-section),  time  (time-series)  or  in  an 
absolute/relative  manner),  (d)  input  (‘means’)  or  output  (‘ends’) 
focus,  (e)  clarity  and  simplicity,  (f)  data  availability  across  time  and 
space,  (g)  flexibility  (e.g.  for  change).  These  dimensions  rather  help 
to  classify  and  evaluate  C&I  measurements  than  to  select  appro¬ 
priate  C&I.  However,  since  they  address  C&I  measurements,  they 
can  be  summarized  into  the  requirements  practicality,  simplicity 
and  reliability,  whereas  the  latter  covers  the  reproducibility  of 
assessment  results. 

According  to  Baker  et  al.  [45]  C&I  should  be:  (a)  able  to  dis¬ 
criminate  among  alternatives,  (b)  complete  in  regard  to  all  goals, 
(c)  operational  and  meaningful  to  understand  the  impacts  of  the 
alternatives,  (d)  non-redundant  and  (e)  few  and  manageable  in 
number.  In  other  words  C&I  should  measure  important  issues, 
comprehensively  and  in  a  meaningful  way,  hence  they  should  be 
relevant  keeping  the  problem’s  goal  in  mind.  Comparability  must 
be  ensured,  as  well  as  a  high  degree  of  independency  of  C&I. 
The  notion  that  C&I  should  be  few  in  number  to  increase  under- 
standability  is  rather  an  overall  requirement  than  an  individual 
selection  requirement  to  decide,  whether  or  not  a  criterion  and  its 
indicator(s)  should  be  chosen. 

For  another  project  [46]  sustainability  C&I  were  selected  and  at 
a  later  stage  reviewed  by  stakeholders  based  on  the  following 
specifications:  (a)  comprehensiveness,  (b)  applicability  to  each 
alternative,  (c)  transparency,  (d)  tractability  of  data,  and  (e) 
practicability  with  respect  to  available  time  and  resources. 
The  first  two  specifications  can  be  summarized  into  the  require¬ 
ment  relevance  and  the  last  two  into  practicality.  Transparency  is 
described  as  the  stakeholders’  ability  to  understand  C&I  or  in  other 
terms  it  describes  the  simplicity  of  C&I.  This  simplicity  of  C&I  is 
also  emphasized  by  Olsthoorn  et  al.  [11]  (p.457)  who  suggest  as  a 
general  guideline  that  C&I  “should  be  as  simple  as  possible  and 
only  as  complex  as  necessary”. 

Gilmour  et  al.  [18]  interviewed  key  stakeholders  individually  to 
verify  and  refine  pre-selected  C&I  with  reference  to  the  specific 
drivers,  aspirations  and  objectives  of  a  specific  sustainable  water¬ 
front  development.  In  this  process  each  criterion  and  its  indicator 
(s)  were  addressed  to  verify  relevance. 

Another  participatory  approach  was  taken  by  Fraser  et  al.  [10] 
who  held  community  focus  group  meetings  to  rank  C&I  according 
to  their  perceived  accuracy  and  ease  of  use.  These  two  require¬ 
ments  refer  to  practicality  and  simplicity  of  C&I,  respectively.  C&I 
can  also  be  selected  by  simply  considering  the  availability  of  data, 
i.e.  the  practicality  of  C&I  can  be  regarded  as  the  single  require¬ 
ment  [13]. 

Lattimore  et  al.  [2]  in  their  paper  about  sustainable  wood  fuel 
production  outline  that  C&I  should  be  as  broadly  applicable  as 
possible  and  be  imbedded  within  a  framework  of  local  adaptability. 
Furthermore,  they  identified  C&I  based  on  comprehensiveness  and 
effectiveness.  These  aspects  can  be  covered  by  the  requirements 
practicality  and  relevance. 


2.2.2.  Requirements  for  C&I  selection:  energy  and  bioenergy  sectors 
Apart  from  literature  about  sustainability  assessments  in  gen¬ 
eral,  requirements  for  C&I  selection  have  also  been  described  in 
literature  specific  to  the  fields  of  energy  and  bioenergy.  In  a  survey 
about  sustainability  C&I  for  bioenergy  systems  [19],  experts  were 
asked  to  rate  C&I  according  to  relevance,  practicality,  reliability  and 
importance.  Since  determining  the  importance  of  C&I  is  typically 
covered  by  separate  decision  making  processes  (e.g.  [47-49],  this 
requirement  was  excluded  for  further  considerations  in  the  C&I 
selection  process  described  in  this  paper.  C&I  relevance  is  the  first 


requirement  described  and  refers  to  relevance  in  regard  to  the 
concept  of  sustainable  bioenergy  systems,  as  well  as  the  contribu¬ 
tion  of  this  aspect’s  assessment  to  an  improved  understanding  of 
the  sustainability  context.  In  terms  of  practicality,  experts  were 
asked  in  the  case  study  described  by  Buchholz  et  al.  [19]  to  consider, 
whether  existing  scales  and/or  measurement  units  and  measure- 
able  threshold  values  exist  for  the  individual  C&I.  In  addition,  the 
ease  of  obtaining  data  and  measuring  C&I  in  a  cost,  time  and/or 
resource  effective  manner  are  mentioned  attributes  for  this  require¬ 
ment.  How  reliable  C&I  assessment  results  are  and  whether  these 
results  can  be  reproduced,  are  considerations  in  regard  to  the 
reliability  requirement.  Further  considerations  for  this  specific 
requirement  are:  high  uncertainty  attached  to  C&I  and  the  ease  of 
reaching  consensus. 

Wang  et  al.  [24]  state  that  C&I  development  and  selection 
requires  parameters  associated  with  reliability,  appropriateness, 
practicality  and  limitations  of  measurement.  In  this  context, 
appropriateness  refers  to  C&I  relevance  and  limitations  of  mea¬ 
surement  could  be  part  of  the  practicality  requirement  for  C&I. 
In  general  the  authors  share  the  opinion  that  a  low  quantity  of 
C&I  is  beneficial  to  the  evaluation  of  energy  systems  process. 
Furthermore,  they  establish  principles  for  decision-makers  to 
select  C&I.  The  following  principles  are  suggested:  (a)  consistency, 
(b)  independency,  (c)  measurability,  (d)  comparability.  The  first 
principle  suggests  consistency  with  the  decision-making  objective 
and  therefore  relevance.  The  independency  principle  demands 
a  criterion  and  its  indicator(s)  to  be  excluded,  if  an  inclusion 
relationship  at  the  same  level  exists.  Furthermore,  a  criterion  and 
its  indicator(s)  should  allow  reflection  of  the  alternatives’  perfor¬ 
mances  from  a  different  perspective.  The  third  principle  requires 
C&I  to  be  quantitatively  or  qualitatively  measurable  and  hence  to 
be  practical.  Finally,  the  comparability  principle  should  ensure 
distinction  between  alternatives,  whereas  it  also  suggests  normal¬ 
ization  processes  to  compare  between  C&I  results. 

In  a  paper  about  sustainability  indicators  in  the  electricity  sector 
[12]  15  selection  requirements  are  grouped  under  three  major 
topics:  (a)  conception,  (b)  application  and  (c)  consistency.  The  first 
requirement  in  the  group  ‘conception’  is  adhesion,  which  is  about  a 
direct  relationship  of  C&I  to  the  analyzed  aspect.  This  requirement 
could  fall  under  the  relevance  requirement.  Feasibility  and  validity, 
the  next  two  requirements  in  this  group  comprise  of  aspects 
covering  practicality  or  reliability  issues  of  C&I.  With  respect  to 
practicality,  data  availability  and  the  feasibility  of  additional  data 
collection  is  also  outlined  in  a  paper,  which  provides  energy 
indicators  for  sustainable  development  [3[.  The  requirement  clarity 
in  Rovere  et  al.  [12]  is  subject  to  the  group’s  last  requirement 
simplicity.  In  the  next  group  ‘application’  the  requirements:  sensi¬ 
tivity  and  temporality  and  ease  cover  the  practicality  aspect  of  C&I, 
because  they  represent  their  ability  for  trend  and  temporal  analyses 
and  the  difficulty  level  of  their  practical  application.  The  spatiality 
requirement  covers  comprehensiveness  of  C&I  and  hence  their 
relevance.  The  remaining  requirement  in  this  group  is  reliability 
and  is  described  as  the  ability  to  capture  both,  positive  and  negative 
issues  in  an  unbiased  manner.  In  the  final  group  ‘consistency’  the 
requirement  discernment  reflects  an  aspect  of  comparability, 
whereas  equilibrium  refers  to  the  independency  of  C&I.  Addition¬ 
ally,  the  requirement  verifiability  comprises  of  the  capacity  of  C&I  to 
reproduce  data  and  therefore  addresses  an  aspect  of  reliability. 
Relevance  is  separately  represented  in  this  group  as  the  remaining 
requirement.  This  requirement  is  also  addressed  in  a  paper  about 
technological  energy  priorities  and  the  multi-criteria  decision¬ 
making  approach  [25],  in  which  the  problem’s  formulation,  a 
countiy’s  specific  energy  characteristics,  as  well  as  its  development 
needs  and  perspectives  are  considered  as  determinants  influencing 
C&I  and  their  performances.  These  determinants  could  be  summar¬ 
ized  into  the  relevance  requirement. 
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3.  Application  of  methodology  in  case  study  area 

3.1.  Scenario  description 

Two  scenarios,  ‘Al-2’  and  ‘A3-4’,  each  with  a  centralized  and  a 
decentralized  bioenergy  strategy  for  the  Scottish  Region  Tayside 
&  Fife  were  developed  (Table  2).  However,  the  scale  of  energy 
generation  differed  between  the  scenarios,  as  well  as  the  alter¬ 
natives.  For  both  of  the  first  scenario’s  alternatives  the  total 
electrical  outputs  were  assumed  to  be  100  MWe.  Also,  a  second 
scenario  with  10  MWe  electrical  outputs  for  each  of  this  scenario’s 
alternatives  was  developed.  For  this  second  scenario  the  diversify¬ 
ing  parameters  and  hence  the  complexity  level  of  comparison  was 
significantly  reduced  by  assuming  the  same  source  and  type  of  fuel 
to  be  used  for  both  alternatives.  The  reason  for  using  these  specific 
scenarios  and  alternatives  for  the  research  case  study  was  to  allow 
the  decision  making  process  to  be  independent  not  just  from  the 
scale  of  the  bioenergy  developments,  but  also  from  the  complexity 
level  of  the  comparison.  In  other  words  by  establishing  and 
presenting  to  bioenergy  experts  scenarios  with  varying  levels  of 
differences  in  specifications  -  one  in  which  all  specifications  differ 
and  one  in  which  only  a  single  specification  distinguishes  the  two 
alternatives  -  the  decision  making  processes  were  less  influenced 
by  the  complexity  of  the  comparison.  Furthermore,  the  presented 
range  of  scenarios  and  alternatives  was  intended  to  demonstrate 
the  broad  application  potential  of  this  participatory  approach  to 
select  sustainability  C&l  in  the  bioenergy  field. 

3.2.  Requirements  for  C&I  selection  for  case  study 

Based  on  the  literature  review  described  previously  (Section 
2.2),  the  final  set  of  requirements  for  C&l  selection  for  the  case 
study  was  chosen  as  shown  in  Table  3.  Each  requirement  was 


described  for  clarification  based  on  the  literature  review.  In  line 
with  this  set  of  requirements,  C&I  were  pre-selected  first  and 
then  presented  to  regional  bioenergy  experts  to  decide  which 
sustainability  C&I  should  be  employed  for  the  case  study. 

3.3.  Pre-selection  of  C&I  for  case  study 

Based  on  the  identified  sustainability  C&I  (Section  2.1)  and 
the  selection  requirements  (Section  2.2),  a  number  of  sustain¬ 
ability  C&I  were  pre-selected  prior  to  presenting  the  resulting 
C&I  short-list  to  regional  bioenergy  stakeholders  to  reach 
consensus  on  them.  For  this  step  the  case  study  scenarios  as 
described  previously  had  to  be  kept  in  mind  at  each  stage  of 
the  process.  The  process  consisted  of  a  series  of  sorting  steps 
before  the  most  appropriate  C&I  were  chosen. 

First,  C&I  from  literature  were  sorted  by  the  four  categories: 
environmental,  economic,  technical  and  social.  As  a  second  sort¬ 
ing  step,  the  C&I  were  summarized,  resulting  in  the  following 
classification: 

-  Environmental:  GHG  Emissions,  Air  Quality,  Water,  Waste  and 
Other  Environmental  C&I. 

-  Economic:  Economic  Viability,  Regional  Energy  Self-Sufficiency 
and  Other  Economic  C&I. 

-  Technical:  Efficiency,  Technology,  Other  Technical  C&I. 

-  Social:  Regional  Job  Creation,  Energy  for  Households,  Employ¬ 
ment  Conditions,  Regional  Food  Security  and  Other  Social  C&I. 

At  this  point  it  became  apparent  that  some  C&I  were  redundant 
and  others  could  be  classified  in  more  than  one  category.  For 
example,  due  its  nature  and  potential  effects  on  both  aspects,  C&I 
related  to  fuel  and  resource  availability  and  import  dependency  can 
either  be  in  the  economic  or  in  the  social  category.  C&I  definitions,  if 


Table  2 

Case  study  scenarios  and  alternatives. 


Alternative 

Total  electrical  output 
(MWe) 

Strategy 

Type  of  fuel 

Source  of  fuel 

Scenario  1 

Al:  Large-scale  bioenergy  plant  (predominantly  for 
electricity  generation) 

100 

Centralization 

strategy 

Single 

location 

Woody  biomass 

Predominantly  from 
abroad 

A2:  20  Medium-scale  bioenergy  combined  heat  and 
power  (CHP)  plants 

100 

De-centralization 

strategy 

Multiple 

locations 

Any  biofuel  (and  related 
technologies) 

Predominantly 
regional  sources 

Scenario  2 

A3:  Medium-scale  bioenergy  combined  heat  and 
power  (CHP)  plant 

10 

Centralization 

strategy 

Single 

location 

Any  biofuel  (and  related 
technologies) 

Predominantly 
regional  sources 

A4:  10  Small-scale  bioenergy  combined  heat  and 
power  (CHP)  plants 

10 

De-centralization 

strategy 

Multiple 

locations 

Any  biofuel  (and  related 
technologies) 

Predominantly 
regional  sources 

Table  3 

Requirements  for  C&I  selection  for  the  case  study. 


Requirement  for  Description:  Source: 

selection: 


Relevance: 

Practicality: 

Reliability: 

Independency: 

Comparability: 

Simplicity: 


How  relevant  are  the  criterion  and  its  indicator(s)  for  decision-making  for  sustainable  bioenergy  developments?  Does 
the  assessment  contribute  to  a  better  understanding  of  the  sustainability  of  bioenergy  developments? 

Do  scales  and/or  measurement  units  exist?  Can  data  easily  be  obtained  and  measured  in  a  cost,  time  and/or  resource 
effective  manner? 

How  reliable/reproducible  are  the  assessment  results?  Is  there  a  high  uncertainty  attached  to  the  assessment  results? 
Are  the  criterion  and  its  indicator(s)  independent  enough,  i.e.  do  they  reflect  the  performance  of  alternatives  from  a 
different  viewpoint  or  do  they  duplicate  other  C&I? 

Are  the  criterion  and  its  indicator(s)  able  to  discriminate  among  alternatives  and  do  they  support  the  performance 
comparison  of  alternatives? 

Are  the  criterion  and  its  indicator(s)  easy  to  understand  by  all  stakeholders? 


[2,12,16,18,19,24,25,45,46] 

[2,3,10-13,16,19,21,24,46] 

[12,16,19,21,24] 

[12.16.24.45] 

[12.24.45] 

[10.21.46] 
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available  from  the  reviewed  papers,  were  of  assistance  to  eliminate 
redundant  C&l  or  to  aggregate  similar  C&I  such  as  ‘Soil  Protection’ 
and  ‘Soil  Quality’  or  ‘Job  Creation’,  ‘Employment  Generation’  and 
‘Contribution  to  Employment  Opportunities’  Creation'.  This  process 
was  characterized  by  selecting  the  most  case-study  relevant  criter¬ 
ion  and  indicator  to  represent  similar  aspects  (e.g.  ‘Air  quality'  and 
the  indicator  ‘Non-GHG  Emissions’).  In  this  procedure  the  six 
requirements  for  C&I  selection  as  defined  in  Table  3  were  also 
considered.  For  example  the  indicator  ‘Net  Present  Value’  was 
selected  to  represent  ‘Economic  Viability’,  because  cost  indicators 
such  as  ‘Investment  Cost’  were  basically  included  in  the  calculation 
of  that  indicator.  At  this  stage,  C&I  were  also  modified  or  new  C&I 
were  developed  to  take  account  of  topical  issues,  which  were  felt  to 
be  not  or  not  fully  addressed  in  the  reviewed  literature  such  as  for 
example  the  indicator  ‘Threat  for  Food  Security',  which  emphasizes 
and  combines  issues  in  regard  to  food  security  and  land  conversion 
caused  by  biofuel  plantation.  These  topical  issues  were  typically 
characterized  by  a  high  relevance  in  regard  to  decision-making  for 
sustainable  bioenergy  developments  at  the  time  when  the  case 
study  was  undertaken.  Also,  in  order  to  add  more  case-study- 
specific  relevance,  indicator  units  were  modified  (e.g.  ‘Created  Jobs/ 
kW  h  (plant  and  supply  chain  specific)’  for  the  criterion  ‘Job 
Creation’).  Based  on  these  changes  the  final  C&I  descriptions  were 
developed.  By  again  comparing  each  of  the  C&I  against  the  six 
requirements  for  selection  as  defined  previously  (Table  3),  the  final 
pre-selection  of  C&I  was  undertaken.  In  this  process,  a  criterion  and 
its  indicator  were  assessed  using  a  three-point  scale  with  Tow', 
‘medium’  and  ‘high’  scores.  The  threshold  score  was  the  score  Tow', 
i.e.  if  a  score  against  a  requirement  was  Tow’,  the  assessed  criterion 
and  its  indicator  was  excluded.  Table  4  below  illustrates  the  scoring 
results  of  the  C&l  pre-selection  process: 

The  following  two  examples  illustrate  how  C&I  were  scored  at 
this  pre-selection  stage: 

3.3.1.  Air  quality 

The  first  scoring  against  the  six  selection  requirements  com¬ 
prised  the  criterion  ‘Air  Quality’  and  its  indicator  ‘Non-GHG 
Emissions’.  This  criterion  and  its  indicator  were  considered  to  be 
highly  relevant  and  to  have  a  high  comparison  level  for  the  case 
study  alternatives.  The  high  score  against  ‘comparability’  was  due 
to  the  quantitative  nature  of  the  criterion’s  indicator,  which 
enables  alternatives  to  be  compared  relatively  easily.  The  high 
score  for  relevance  was  justified  by  the  aggregation  potential  of 
the  criteria,  as  it  allows  the  inclusion  of  several  emissions 
(NOx,  CO,  S02  and  particulates).  In  regard  to  ‘practicality’, 


‘reliability’,  ‘independency’  and  ‘simplicity’  ‘medium’  scores  were 
assigned.  Although,  often  measured  for  projects  of  this  type,  the 
indicator  ‘Non-GHG  emissions’  and  its  mentioned  aggregation 
potential  was  regarded  as  causing  potential  limitations  in  regard 
to  the  first  two  requirements,  because  data  availability  and 
reliability  of  all  the  case  study’s  emissions  was  assumed  to  be 
incomplete  or  limited.  Aggregation  was  also  the  reason  for  the 
medium  score  for  ‘simplicity’,  because  multiple  data  for  one 
indicator  was  regarded  as  influencing  negatively  the  understand- 
ability  of  the  criterion  and  its  indicators  for  all  bioenergy  experts. 
The  criterion  and  its  indicator  are  influenced  by  several  other  C&I 
and  thus  a  ‘medium’  score  was  assigned  to  the  requirement 
‘independency’. 

3.3.2.  Regional  food  security 

The  criterion  ‘Regional  Food  Security’  with  its  indicator  ‘Threat 
to  Food  Security’  scored  ‘high’  against  ‘relevance’,  because  this 
aspect  was  related  to  both,  the  topical  food  versus-fuel-contro- 
versy,  which  covers  the  issue  of  diverting  farmland  or  crops  for 
biofuels  production  in  detriment  of  the  food  supply,  as  well  as  the 
already  existing  and  predicted  increase  in  competition  for  biofuels 
resources  in  the  case  study  area.  It  was  assumed  that  it  would  be 
difficult  to  obtain  and  measure  data  in  a  cost,  time  and/or  resource 
effective  manner  and  predictive  data  would  rather  be  uncertain, 
due  to  the  early  stage  of  energy  crop  plantation  in  Scotland.  Hence, 
for  both  requirements  ‘practicality’  and  ‘reliability’  ‘medium’  score 
were  assigned.  The  criterion  is  influenced  by  other  C&I  and  scored 
‘medium’  against  ‘independency’.  The  criterion  allows  discrimina¬ 
tion  among  the  scenarios’  alternatives  and  therefore  scored  ‘high’ 
against  ‘comparability’.  For  the  last  requirement  ‘simplicity’  it  was 
assumed  that  not  all  experts  would  easily  understand  the  criterion 
and  its  indicator,  which  was  reflected  in  a  ‘medium’  score. 

Table  5  below  illustrates  the  pre-selected  C&I.  This  table  was 
presented  to  bioenergy  experts  for  the  final  selection  process: 


3.4.  Final  selection  of  criteria  and  indicators  by  experts  in  case 
study  area 

Similar  to  other  studies  in  the  sustainability  and  bioenergy 
sector  [19,50],  participants  of  the  multi-stakeholder  forum  were 
bioenergy  experts  rather  than  members  of  the  general  public  and 
like  in  Sheppard  and  Meitner  [1]  were  selected  based  on  their 
knowledge  and  expertise  in  the  subject,  their  history  with  plan¬ 
ning  processes  and  based  on  how  much  they  would  be  affected  by 


Table  4 

Scoring  results  of  the  C&I  pre-selection  for  the  case  study. 


Category 

Criterion 

Indicator 

Relevance 

Practicality 

Reliability 

Independency 

Comparability 

Simplicity 

Environmental 

Air  quality 

Non-GHG  emissions 

High 

Medium 

Medium 

Medium 

High 

Medium 

GHG  emissions 

C02  emission 

High 

Medium 

Medium 

Medium 

High 

High 

Water 

Water  management 

Low 

Medium 

High 

Medium 

High 

High 

Waste 

Waste  re-processed 

High 

Medium 

Medium 

Medium 

High 

High 

Others 

Others 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Economic 

Economic  viability 

Net  present  value 

High 

Medium 

Medium 

Medium 

High 

Medium 

Regional  energy  self-sufficiency 

Fuel  import  dependency 

High 

Medium 

Medium 

Medium 

High 

Medium 

Others 

Others 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Technical 

Efficiency 

Energy  efficiency 

High 

High 

High 

Medium 

High 

High 

Technology 

Knowledge  of  the  technology 

High 

Medium 

Medium 

High 

High 

Medium 

Others 

Others 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Social 

Regional  job  creation 

Job  creation 

High 

Medium 

Medium 

Medium 

High 

High 

Energy  for  households 

Number  of  households  supplied 

Low 

Low 

Low 

Medium 

Medium 

High 

Employment  conditions 

Average  level  of  job  income 

Low 

Medium 

High 

Medium 

Medium 

High 

Regional  food  security 

Threat  to  food  security 

High 

Medium 

Medium 

Medium 

High 

Medium 

Others 

Others 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 
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Table  5 

Pre-selected  C&I  for  the  case  study. 


Category 

Criterion 

Indicator 

Unit 

Description 

Environmental 

GHG  emissions 

C02  emissions 

kg/kW  h  (supply  chain  specific) 

Emissions  of  C02  across  bioenergy  supply  chain 

Air  quality 

Non  GHG- 

emissions 

kg/kW  h  (operational  plant  specific) 

Emissions  of  NOx,  CO,  S02  and  particulates  from  operational  bioenergy 
plant(s) 

Waste 

Waste  re¬ 
processed 

%  of  Scottish  waste  re-processed  to 
fuel  (per  annum) 

Waste  re-processed  to  fuel  for  bioenergy  production 

Economic 

Economic  viability 

Net  present 
value 

£/kW  h  (operational  plant  specific) 

Expected  incoming  and  outgoing  cash  flows  derived  to  present  economic 
values  to  enable  a  comparison  independent  of  time 

Regional  energy 
self-sufficiency 

Fuel  import 
dependency 

%  of  imported  feedstock  (operational 
plant  specific)  (per  annum) 

Percentage  of  imported  feedstock  for  the  bioenergy  plant(s) 

Technical 

Efficiency 

Energy 

efficiency 

%  (operational  plant  specific) 

Expresses  the  technology’s  ability  to  convert  the  primary  energy  source 
to  energy 

Technology 

Knowledge  of 
technology 

%  (worldwide) 

Represents  the  technology’s  maturity  rate,  as  well  as  its  uptake/adoption 
in  the  international  market 

Social 

Regional  job 
creation 

Job  creation 

Created  jobs/kW  h  (plant  and  supply 
chain  specific) 

Number  of  direct  and  indirect  jobs  created  at  the  bioenergy  plant(s)  and 
along  the  Scottish  bioenergy  supply  chain 

Regional  food 

Threat  to  food 

%  of  total  arable  land  used  for  energy 

Reflects  the  availability  of  remaining  arable  land  for  food  production  in 

security 

security 

crop  plantation 

Scotland 

the  scenarios’  alternatives.  It  was  ensured  that  all  interests  and 
aspects  of  sustainability  in  the  field  were  covered  by  equal 
proportions  of  participating  representatives  from  different  back¬ 
grounds,  which  as  outlined  by  Voinov  and  Bousquet  [51]  was  also 
intended  to  increase  public  acceptance  and  credibility  of  the 
forum’s  outcomes.  A  relatively  balanced  composition  of  partici¬ 
pants  from  local  authorities,  the  regulative  body,  the  business 
support  agency,  environmental  protection,  harvesting  and  supply, 
sawmilling,  bioenergy  production,  agriculture,  forestry  and  waste 
management  was  achieved  by  nonproportional  quota  sampling 
[52].  In  total  13  regional  bioenergy  experts  participated  in  the 
multi-stakeholder  forum. 

The  invited  participants  were  asked  to  reach  consensus  about 
the  final  set  of  sustainability  C&I  for  the  case  study  scenarios  and 
alternatives.  It  was  explained  that  the  selection  of  the  most 
appropriate  C&I  is  crucial  to  assess  sustainability,  because  any 
aspects  not  included  would  simply  be  ignored  from  the  decision 
making  process,  which  again  would  influence  assessment  results. 

First,  a  PowerPoint™  slide  explaining  the  four  distinct  aspects 
within  sustainability  that  interrelate  -  environmental,  economic, 
social  and  technical  -  was  presented  to  the  participants  of  the 
multi-stakeholder  forum.  After  highlighting  the  goal  of  the  multi¬ 
stakeholder  forum  (consensus  about  C&I  selection)  and  explaining 
the  procedure,  the  requirements  for  C&I  selection  along  with  their 
descriptions  (see  Table  3)  were  presented  on  another  Power¬ 
Point™  slide,  of  which  also  hard-copies  were  handed  out. 
The  experts  were  explained  that  those  six  requirements  were 
used  for  pre-selection  and  asked  to  make  their  own  selection 
choices  also  based  on  them.  Furthermore,  presented  slides  with 
problem  hierarchies  for  both  scenarios  allowed  participants  to 
keep  an  overview  of  the  relationships  between  the  goal,  cate¬ 
gories,  criteria  and  alternatives  (see  Fig.  1  as  an  example). 

The  pre-selected  C&I  (Table  5)  were  presented  to  the  partici¬ 
pants  grouped  by  sustainability  aspect.  First,  questions  were 
answered  and  issues  were  clarified  in  regard  to  C&I  descriptions. 
Then,  the  process  of  final  C&I  selection  itself  was  carried  out  by 
directly  asking  the  forum  participants  to  reach  consensus  about 
the  suitability  of  each  pre-selected  criterion  and  its  indicator  to 
assess  the  case  study’s  scenarios  and  alternatives.  The  participants 
were  encouraged  to  confirm  or  make  changes  to  the  presented  set 
or  propose  new  C&I,  again  by  taking  account  of  the  defined  six 
requirements  for  selection.  In  cases  when  consensus  about  pre¬ 
selected  or  newly  proposed  C&I  could  not  be  reached,  the  C&I  were 
regarded  suitable,  when  there  was  evidence  that  a  significant 
majority  of  forum  participants  agreed  on  them.  To  keep  track  of 
the  communication  notes  were  taken  during  and  after  the  event. 


4.  Results  and  discussion 

The  processes  of  the  pre-  and  final  selection  of  C&I  are  discussed 
in  the  following. 

4.1.  C&l  pre-selection  process 

When  reflecting  on  the  pre-selection  process  it  can  be  observed 
that  all  pre-selected  C&I  scored  ‘high’  against  the  requirements 
‘relevance’  and  ‘comparability’.  The  reason  for  that  was  that 
throughout  the  process  the  case  study  scenarios  and  alternatives 
were  kept  in  mind.  In  other  words,  the  steps  including  sorting  and 
classification,  modifying  of  indicator  units  and  the  final  descrip¬ 
tions  of  C&I  contributed  to  increased  ‘relevance’  and  ‘comparabil¬ 
ity’  of  those  C&I.  Scoring  against  the  requirements  ‘practicality’ 
and  ‘reliability’  was  highly  subjective,  because  the  scores  were 
based  on  the  most  assumptions  and  uncertainties  in  regard  to  data 
gathering,  measuring  and  reliability.  When  scoring  some  C&l 
against  those  requirements,  location-dependent  levels  of  data 
availability  and  quality  had  to  be  considered,  which  added  more 
complexity  to  the  scoring  processes.  In  this  connection  it  had  to  be 
kept  in  mind,  that  the  described  scoring  processes  had  the  sole 
purpose  to  pre-select  C&I.  As  described  previously,  participants  of 
the  multi-stakeholder  forum  had  the  opportunity  to  confirm, 
change  or  propose  new  C&I  resulting  in  a  final  set.  When  scoring 
‘simplicity’  of  C&I,  the  groups/organizations  and  positions/depart¬ 
ments  of  the  participants  invited  to  the  multi-stakeholder  forum 
were  taken  into  account.  However,  the  assumptions  made  when 
scoring  C&I  against  this  selection  requirement  can  still  be  regarded 
as  highly  subjective,  because  knowledge  and  understanding  of  C&I 
first  and  foremost  depend  on  personal  backgrounds  of  individual 
forum  participants. 


4.2.  Participatory  selection  of  environmental  C&l 

In  the  multi-stakeholder  forum,  in  which  the  final  C&I  took  place, 
participants  suggested  that  C&I  covering  biodiversity  and  visual 
impacts  of  potential  plant(s)  could  be  included.  However,  after  a 
discussion  the  forum  participants  ruled  these  aspects  out,  because  of 
the  research’s  scope  and  focus  on  supporting  sustainability  decision¬ 
making  to  screen  and  assess  location  areas,  rather  than  specific  site 
locations,  for  which  these  aspects  are  typically  assessed.  For  the  same 
reason,  the  suggestion  of  the  regulator’s  representative  to  look  at  the 
supply  chain,  including  road  transport  of  fuel  when  assessing  the 
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Problem  Hierarchy  -  Scenario 


GOAL:  Regional  Sustainable  Bioenergy 
Generation 


ENVIRONMENTAL 
criteria  (EN) 


ECONOMIC 
criteria  (EC) 


TECHNICAL 
criteria  (T) 


SOCIAL  criteria 
(S) 


r 

ALTERNATIVE  1:  Large-Scale 

"  A 

ALTERNATIVE  2:  Medium-Scale 

Bioenergy  Generation 

Bioenergy  Generation 

Fig.  1.  Problem  hierarchy  for  scenario  1. 


criterion  ‘air  quality’  was  not  found  suitable.  Thus,  it  was  agreed  that 
although  assessing  the  three  aspects  is  important,  this  would  have  to 
be  carried  out  after  identifying  concrete  plant  locations. 

Contrary  to  that  the  criterion  ‘GHG  emissions’  and  its  indicator 
‘C02  emissions’  were  confirmed  by  the  majority  of  the  forum 
participants,  although  it  includes  assessment  of  C02  emissions  across 
the  Scottish  supply  chain.  Forum  participants  from  the  environmen¬ 
tal  protection  organization  and  the  bioenergy  producers  highlighted 
that  when  assessing  this  criterion  and  its  indicator  the  bioenergy 
supply  chain  has  to  be  defined  by  keeping  the  goal  of  the  decision 
making  problem,  its  scenarios  and  alternatives,  as  well  as  the  ‘Life 
Cycle  Assessment’  (LCA)  approach  and  the  EU  Commission’s  ‘Report 
on  sustainability  requirements  for  the  use  of  solid  and  gaseous 
biomass  sources  in  electricity,  heating  and  cooling’  [53]  in  mind.  It 
was  also  proposed  and  agreed  by  most  experts  to  extent  the  scope  of 
the  indicator  to  ‘C02— equivalent  GHG  emissions’. 

Suggestions  for  change  and  improvement  were  also  made 
for  the  criterion  ‘waste’  and  its  indicator  ‘waste  re-processed’. 
Although  acknowledging  the  potential  for  Greenhouse  Gas  (GHG) 
emissions  savings,  the  forum  participant  from  the  large  scale 
bioenergy  producer  questioned  the  economic  viability  of  re¬ 
processing  waste  materials  into  biofuel.  On  the  other  hand,  the 
participant  proposed  to  introduce  an  indicator  to  measure  residual 
ash  usage,  on  which  the  forum’s  participants  agreed.  However,  in 
terms  of  re-processing  waste  materials  into  biofuel,  most  of  the 
forum  participants  argued  that  its  use  would  be  the  individual 
business  decision  of  the  bioenergy  producer  and  that  the  eco¬ 
nomic  success  of  waste  re-processor  companies  in  Scotl¬ 
and  demonstrated  that  waste  re-processing  can  be  profitable. 
In  principle,  it  could  be  assumed  that  the  reasoning  of  the  forum 
participant  representing  the  large-scale  bioenergy  producer  was 
motivated  to  a  certain  extent  by  the  organization’s  plans  to  build  a 
large-scale  bioenergy  plant  in  the  research  case  study  area  in  the 
future.  For  this  plant  it  is  planned  to  utilize  only  small  amounts  of 
re-processed  waste  as  fuel  and  to  put  processes  in  place  to  make 
residual  ashes  useable. 


Also  covering  the  waste  aspect,  a  forum  participant  from  a 
business  support  organization  brought  forward  that  the  indicator 
and  its  description  could  be  more  specific  in  terms  of  waste  type 
and  pointed  out  that  in  re-processing  waste  different  classifica¬ 
tions  and  regulations  for  various  materials  have  to  be  kept  in 
mind.  However,  it  can  be  argued  that  breaking-down  the  several 
re-processed  waste  types  could  become  relatively  complex  and 
less  practical  compared  to  measuring  a  single  indicator,  which 
accumulates  all  types  of  re-processed  waste.  This  is  particularly 
true  when  taking  into  account  that  for  three  of  the  four  alter¬ 
natives  of  the  case  study  scenarios  utilization  of  any  type  of 
biomass  feedstock  and  related  technologies  was  possible.  For  this 
reason,  the  criterion  and  its  indicator  were  not  changed. 

4.3.  Participatory  selection  of  economic  CEfl 

The  forum  participants  from  a  sawmill,  a  bioenergy  producer  and 
a  business  support  organization  raised  concerns  about  the  availability 
of  biomass  feedstock  supplies  and  the  increasing  dependency  on 
biofuel  imports.  They  also  made  aware  of  possible  consequences  of 
not  securing  alternative,  sustainable  supply  sources.  Therefore,  along 
with  the  substantial  majority  of  the  other  experts,  they  agreed  on  the 
particular  usefulness  of  the  proposed  criterion  ‘regional  energy 
self-sufficiency’  and  its  indicator  ‘fuel  import  dependency’  covering 
this  aspect. 

Moreover,  in  addition  to  the  pre-selected  economic  C&I  two 
new  aspects  were  suggested  to  be  incorporated  during  this  stage 
of  the  multi-stakeholder  forum.  One  aspect  was  about  covering 
future  economic  incentives  for  the  bioenergy  sector  and  the  other 
aspect  about  addressing  the  level  of  regulations  for  the  different 
alternatives  of  the  case  study  scenarios.  However,  it  was  found 
that  both  proposed  aspects  are  directly  or  indirectly  covered  by 
the  pre-selected  criterion  ‘economic  viability’  and  its  indicator  ‘net 
present  value  (NPV)’,  as  they  either  generate  additional  income  or 
result  in  costs  to  maintain  or  put  measures  into  place  to  comply 
with  regulation. 
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Table  6 

Final  set  of  selected  C&l  for  the  case  study. 


Category 

Criterion 

Indicator 

Unit 

Description 

Environmental 

GHG  emissions 

C02- 

equivalent 

GHG 

emissions 

kg/kW  h  (supply  chain  specific) 

C02-equivalent  GHG  emissions  across  bioenergy  supply  chain 

Air  quality 

Non  GHG- 

emissions 

kg/kW  h  (operational  plant 
specific) 

Emissions  of  NOx,  CO,  S02  and  particulates  from  operational 
bioenergy  plant(s) 

Waste 

Waste  re¬ 
processed 

%  of  Scottish  waste  re-processed  to 
fuel  (per  annum) 

Waste  re-processed  to  fuel  for  bioenergy  production 

Economic 

Economic  viability 

Net  present 
value 

£/kW  h  (operational  plant  specific) 

Expected  incoming  and  outgoing  cash  flows  derived  to  present 
economic  values  to  enable  a  comparison  independent  of  time 

Regional  energy  self- 
sufficiency 

Fuel  import 
dependency 

%  of  imported  feedstock 
(operational  plant  specific)  (per 
annum) 

Percentage  of  imported  feedstock  for  the  bioenergy  plant(s) 

Technical 

Efficiency 

Energy 

efficiency 

%  (operational  plant  specific) 

Expresses  the  technology’s  ability  to  convert  the  primary  energy 
source  to  energy  (electricity  and/or  thermal  energy)  for  usage 

Technology 

Knowledge  of 
technology 

%  (worldwide) 

Represents  the  technology’s  maturity  rate  and  its  uptake/adoption 
in  the  international  market,  which  reflects  the  technology's 
reliability 

Social 

Regional  job  creation 

Job  creation 

Created  jobs/kW  h  (plant  and 
supply  chain  specific) 

Number  of  direct  and  indirect  jobs  created  at  the  bioenergy  plant 
(s)  and  along  the  Scottish  bioenergy  supply  chain 

Regional  food  security/ 

Land  use 

%  of  total  productive  land  use 

Reflects  the  change  of  landscape  and  land  use  due  to  energy  crop 

Change  of  landscape  and 
land  use 

change 

change  in  favour  of  energy  crop 
plantation 

plantation  and  the  availability  of  remaining  productive  land  in 
Scotland 

Representatives  of  a  bioenergy  producer  and  a  business  sup¬ 
port  organization  further  suggested  that  economic  C&I  covering 
risk  could  be  considered.  They  argued  that  this  would  address  the 
risks  arising  from  technical  failure  and  biofuel  supply  bottlenecks. 
However,  it  can  be  argued  that  these  risks  are  already  indirectly  or 
directly  addressed  by  the  pre-selected  criteria  ‘regional  energy 
self-sufficiency’  and  ‘technology’  and  their  respective  indicators 
‘fuel  import  dependency’  and  ‘knowledge  of  technology'.  There¬ 
fore,  these  suggestions  for  change  were  not  further  considered. 

4.4.  Participatory  selection  of  technical  C&I 

The  two  representatives  of  the  bioenergy  producers  in  the  multi¬ 
stakeholder  forum  outlined  that  the  technical  criterion  ‘efficiency’ 
and  its  indicator  also  cover  economic  issues  and  could  therefore 
categorized  as  either  technical  or  economic  C&I.  Furthermore,  they 
stated  that  the  criterion  could  be  renamed  to  ‘energy  productivity’ 
to  address  more  the  possible  shared  use  of  generated  heat,  for 
instance  for  district  heating,  for  some  alternatives  of  the  case  study 
or,  alternatively,  the  description  of  this  criterion  and  indicator  could 
be  clearer.  After  the  multi-stakeholder  forum,  it  was  decided  that 
based  on  the  common  use  and  the  definitions  found  in  literature, 
the  criterion  and  its  indicator  remains  in  the  technical  category  and 
renaming  would  not  be  required  for  the  case  study.  However,  in 
order  to  clarify  what  is  meant  by  the  criterion  and  its  indicator  the 
description  was  amended. 

Furthermore,  a  forum  participant  representing  a  bioenergy 
producer  further  proposed  that  the  description  of  the  criterion 
‘technology’  and  its  indicator  ‘knowledge  of  technology’  should  be 
improved,  as  this  aspect  basically  measures  how  save  and  reliable 
a  technology  is. 

4.5.  Participatory  selection  of  social  C&I 

Again  with  respect  to  the  supply  situation,  the  forum  participants 
from  an  environmental  protection  organization  and  a  sawmill,  sug¬ 
gested  to  take  account  of  business  and  jobs  taken  away  from  large 
traditional  industries  in  the  case  study  area,  which  compete  for  the 
same  biofuel  supplies  as  the  bioenergy  industry,  such  as  the  wood 
panel  industry.  However,  it  was  argued  that  availability  of  supplies 
would  just  be  one  of  many  factors  influencing  the  development  of  this 


competitive  market.  Based  on  this  argument,  the  majority  of  experts 
agreed  that  no  practical  and  measurable  criterion  and  its  indicator 
would  cover  the  issue  appropriately. 

Another  forum  discussion  followed  with  forum  participants  from 
the  environmental  protection  organization,  a  local  authority,  a  bioe¬ 
nergy  producer  and  a  forestry  support  organization  suggesting  that 
land  use  or  landscape  change  caused  by  energy  crop  plantation  could 
be  addressed.  As  indicators  ‘marginal  price  for  farmers  to  switch  to 
biofuels’  and  ‘use  of  marginal  land’  were  proposed.  However,  the 
substantial  majority  of  forum  participants  agreed  that  these  indicators 
would  be  relatively  specific  and  that  the  pre-selected  criterion  ‘threat 
to  food  security’  with  its  indicator  and  unit  ‘%  of  total  arable  land  used 
for  energy  crop  plantation’  principally  already  covers  the  aspect  for  the 
given  scenarios  appropriately.  By  renaming  the  criterion  and  indicator, 
as  well  as  by  amending  the  unit  and  description,  this  was  made  more 
transparent  and  understandable. 

4.6.  Final  set  of  selected  C&I 

In  summary  the  set  of  pre-selected  C&l  (Table  5)  was  confirmed  to 
be  appropriate  by  a  substantial  majority  of  the  participants  with  the 
exception  of  the  mentioned  relatively  minor  changes  or  modifica¬ 
tions  of  four  C&I.  Principally,  the  effectiveness  of  the  pre-selection 
process  was  underlined  by  the  few  changes.  Even  in  cases  where  no 
changes  would  be  suggested  by  experts  or  other  stakeholders  in 
similar  research  or  decision  making  situations,  as  mentioned,  follow¬ 
ing  a  participatory  approach  is  vital  to  ensure  acceptance,  transpar¬ 
ency  and  credibility  throughout  the  decision  making  process. 
Furthermore,  this  approach  encourages  ownership  at  a  local  level, 
which  can  be  crucial  when  it  comes  to  the  implementation  phase  of 
bioenergy  developments,  as  local  ownership  could  increase  stake¬ 
holder  acceptance  leading  to  a  higher  probability  of  project  approval 
and  time  savings.  Although  related  to  identified  C&I  such  as 
‘participation’  and  ‘empowerment’  (Table  1),  ‘local  ownership’,  as 
well  as  ‘regional  equity’  also  could  be  considered  as  separate  C&I  for 
future  research  particularly  focusing  on  decentralized  and  centralized 
bioenergy  generation.  Further  notable  is  that  qualitative  C&I  were  not 
suggested  in  the  multi-stakeholder  forum,  which  is  in  line  with  the 
arguments  brought  forward  previously  (Section  3.3).  Table  6  illus¬ 
trates  the  final  set  of  selected  C&I  for  the  case  study  with  the  changes 
highlighted.  In  principle,  it  remains  to  be  seen  whether  the  selected 
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C&I  can  successfully  be  employed  for  decision  making  processes,  if 
the  described  scenarios  and  alternatives  were  implemented  in  the 
case  study  area  or  in  similar  regions.  This  could  be  a  subject  for  future 
research. 

5.  Conclusions 

This  paper  provides  a  generic  approach  for  selecting  sustain¬ 
ability  C&l  for  bioenergy  developments  by  using  a  participatory 
methodology.  Selecting  appropriate  C&I  to  assess  the  sustainability 
of  a  project  or  development  is  crucial  and  significantly  influences 
the  assessment  results. 

In  principle,  the  methodology  consisted  of  two  processes,  a 
pre-selection  of  suitable  C&I  by  the  researchers  and  a  final 
selection  by  regional  bioenergy  experts  in  a  multi-stakeholder 
forum.  The  methodology  was  applied  in  a  Scottish  region  (Tayside 
&  Fife). 

Although  the  participants  of  the  multi-stakeholder  forum  regarded 
the  resulting  set  of  C&I  as  being  suitable,  it  remains  to  be  seen 
whether  the  selected  C&I  can  successfully  be  employed  for  decision 
making  processes,  if  the  described  scenarios  and  alternatives 
were  implemented  in  the  case  study  area.  However,  this  set  of  C&I 
can  serve  as  a  benchmark  set  in  this  and  similar  decision  making 
situations.  For  that  purpose  the  case  study’s  two  scenarios  and  four 
alternatives  were  deliberately  developed  in  a  rather  generic  way 
allowing  a  wide  range  of  application  potential  within  the  energy  and 
particularly  the  bioenergy  field.  For  similar  research  or  decision 
making  situations  this  particular  set  of  C&I  can  be  used  as  a  starting 
point  for  C&I  pre-selection  prior  to  regional  bioenergy  stakeholders  or 
wider  stakeholders  making  final  C&I  selection  decisions  on  a  case-to- 
case  basis. 

Furthermore,  the  literature  reviews  to  identify  relevant  C&I  and 
requirements  for  C&I  selection  provide  a  broad  application  poten¬ 
tial  to  assist  C&I  selection,  as  the  sustainability  field  in  general,  the 
energy  sector  and  the  bioenergy  sector  in  particular  are  covered. 
The  same  applies  for  the  methodology  itself.  Although,  for  other 
research  or  decision  making  situations  different  C&I  may  be 
relevant,  the  presented  approach  to  systematically  select  C&I  and 
to  justify  this  selection  can  be  undertaken. 

The  methodology  for  selecting  C&I  can  be  combined  with 
participatory  C&I  prioritization  to  determine  the  relative  impor¬ 
tance  of  each  criterion  and  its  indicator.  This  can  help  to  improve 
performance  assessments  of  selected  C&I  and  support  participa¬ 
tory  decision  making. 
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